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ABSTRACT 

The subjective magnitude estimation (SME) procedure 
was used to obtain estimates of relative word frequency from two 
adult groups (15 lexicographers, 13 other adults) for 60 words 
ranging widely in objective frequency. Lexicographers rendered more 
reliable estimates, and their averaged data correlated more highly 
with objective log frequency than those of the second group. The 
objective frequency of the first stimulus considered in the SME task 
is not related to a subject's overall accuracy in predicting 
objective frequency, but accuracy is related to the subject's 
tendency to perceive frequency ratios as relatively large. Subjective 
estimates measure something slightly different from what is indexed 
by currently available objective counts, and may be more valid 
measures of true word probability. (Author/CK) 
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Measurement Properties of Subjective Magnitude Estimates of Word Frequency 



John B. Carroll 
Educational Testing Service 

Abstract 

Stevens’ subjective magnitude estimation (SME) method was used in 
obtaining estimates of relative word frequency from two adult groups 
(15 lexicographers, 13 other adults) for 60 words ranging widely in 
objective frequency. Lexicographers rendered more reliable estimates, 
and their averaged data correlated more highly (.970) with objective 
log frequency than those of the second group (»923)« Th e objective 
frequency of the first stimulus considered in the SME task is not 
related to an S's overall accuracy in predicting objective frequency, 
but accuracy is related to the S's tendency to perceive frequency 
ratios as relatively large. Subjective estimates measure something 
slightly different from what is indexed by currently available objective 
counts, and may be more valid measures of true word probability. 




2 



Measurement Properties of Subjective Magnitude Estimates of Word Frequency^ 

John B. Carroll 
Educational Testing Service 

The subjective magnitude estimation (SME) procedure developed by 
Stevens (195 6; 1958, p. 193) has been successfully applied by Shapiro 
(1969) to the scaling of word frequency; Shapiro found correlations 
of from .920 to .958 between subjective estimates and objective data 
on word frequency when both variables were in logarithmic form. The 
purpose of the present study was to examine the reliability of such 
subjective estimates as a function of the number and characteristics 
of the raters. A further purpose was to investigate individual 
differences in the accuracy of subjective estimates (i.e., in the 
correlations of Ss’ judgments with objective data) and in the exponents 
implied by the power-law function. Consideration was given to whether 
these differences are a function of the objective frequency of the 
first word considered in the SME task, in view of the fact that the 
response assigned to that word tends to establish the characteristics 
of the arbitrary scale used by the S in making his subsequent responses. 
There was also interest in whether evidence could be obtained as to 
whether subjective estimates are more valid indices of true word 
probability than currently available objective word-frequency counts. 

Method 

Subjects . Two samples of subjects were used. One (N = 15) was a 
group of individuals who as a consequence of serving on the editorial 
staff of the American Heritage Dictionary (Morris, 19^9) had had extensive 
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experience in lexicography. These lexicographers might "be expected 
to have particularly accurate perceptions of word frequency. The other 
sample (N = 13) was composed of teachers, housewives, and other adults, 
all with college educations, who volunteered for the study. Ss were 
not paid for participation; they were allowed to perform the task at 
their own convenience, but were cautioned not to refer to any objective 
word-count data in making their judgments. 

Materials . The stimuli were 60 words that had been used by 
Shapiro (1969); these are listed in descending order of objective 
frequency in the first column of Table 1. These words had been 
selected by Shapiro to represent a wide range of frequency according 
to two large frequency counts (Thorndike 8c Lorge, 1944; Ku£era 8s Francis, 
1967). In the second column of Table 1, the raw frequencies assembled 
by Shapiro are converted to SFI (Standard Frequency Index) values 
according to the formula proposed by Carroll (in press): 

‘ SFI = 10 (log 1Q f + 4 ), 

where f = frequency per million. The words were printed in three columns 
on computer-output paper in the same random order that had been used 
by Shapiro (1967, p. 194). Associated with each word, however, was 
a computer-generated number to indicate the order in which the words 
were to be considered in the judgment process; the rating form given to 
each S had a different random order of these numbers, ranging from 1 
through 60. 

Procedure . With slight modifications, Shapiro's instructions for 
the subjective magnitude estimation procedure were used. Ss were told 
that their task was to "tell with numbers" how frequently the words 
occur in written English. "Give one of the words any number that seems 
appropriate to you. Then give numbers to the other words so that these 
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numbers give your own impression of the relative frequency of these 
words in written English. Thus, if a word seems 20 times as frequent 
as another, give it a number 20 times as large. If, however, it seems 
only half as frequent, give it a number only one-half as large,... It 
is best to use a first number that is fairly large so that you have 
room to move both up and down. You may use fractions, decimals, or 
whole numbers, but not negative numbers. You will not be timed. Work 
at your own speed. There are no right or wrong answers. We are simply 
interested in your own judgments." After a brief practice in scaling 
the words patron , salad , dictum, and mother, in that order, the S was 
told: "On the accompanying sheet, the 60 words are listed in random 
order, but each word is tagged with a number indicating the order in 
which you are to take up the words. Please follow this order, 
starting with (l), (2), etc., until you reach (6o) . It should not be 
too difficult to skip around to find the order numbers." Ss rfrote their 
responses on blanks placed to the right of each word, and could at 
any time inspect or change the responses they had made. 

Results 

For each S, the following statistical operations were performed: 

(l) The numbers given for each word were converted to logarithms to 
base 10; (2) the mean and S.b. of these log values were computed, and 
the log values were then converted to z-scores so that comparable 
values for the words could be obtained over the different Ss; ( 3 ) the 
correlation between the log values and the SFI values was found, along 

with the corresponding regression equations. 

For each sample of Ss, the following statistics weiv* then computed: 
(l) the mean and S.D. of the mean log values and of the mean z-scores, 
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(2) for each word, the mean log value, the mean z-score value, and the 
S.D. of the z-scores over the Ss; (3) over the 60 words, the correlations 
"between the objective SFI values and (a) the mean log values and (b) the 
mean z-scores, with the corresponding regression equations; (4) reliability 
coefficients for individual and group-average z — scores by the methods of 
Ebel (1951), indicating degree of inter-rater agreement. 

The mean log values, the mean _z-ncore values, and the S.D.'s of 
z-scores are given for each word and for each word in Table 1 . Data are 
also given for combined samples. 

Insert Table 1 about here 

Sample 1 (N = 15 lexicographers) yielded significantly more reliable 
estimates than Sample 2 (N ="• 13 other adults) as determined by several 
criteria. As shown in Table 2, the respective reliability coefficients 

Insert Table 2 about here 

for individual ratings in z-score form were .869 and .746, significantly 
different by Fisher's test (t = 2. CO., df = ll4, p< .05, each correlation 
being based on n = 60 words) . Using the Spearman-Brown prophecy formula 
to boost the obtained reliability for the average z-score ratings of 
Sample 2 (.974) tc make it comparable with that of Sample 1, we obtain 
.978, which is significantly different from the obtained reliability of 
average ratings for Sample 1, .990 (t = 2.16, 2 < -°5). The mean square 
errors for the two groups are also significantly different, .134 and .259 
respectively (F^g g 2 g = 1-94, £< .001). This difference is also 
reflected in the fact that the median S.D. of z-scores over the 60 words 
is significantly lower in Sample 1, . 327> than in Sample 2, .442 (by 
Wilcoxon' s test for non independent samples, z. = 5*04, £ < . 001 ). 
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There is, however, a tendency for the individual Zj-scores to "be 
increasingly unreliable as the SFI values decrease, i.e., as the 
stimulus words become less and less frequent in terms of objective 
data. Table 3 shows average S.D.'s of z-scores by groups of words 
according to SFI values. As seen there, the S.D.'s of z-scores yielded 
by lexicographers tend to be in the neighborhood of .32 for all words 
except those of lowest frequency, average SFI = 37-4, where their average 
S.D. is .482 (Fj. =9-6, £ < . 001 ). In Sample 2 , however, the S.D.'s 
are generally higher than those of the lexicographers, and these S.D.'s 
increase markedly as the SFI values decrease (F^^ = 7 - 12 , £ < . 001 ). 
There was a small yet significant correlation (r = . 272 , df = 57, P < - 05 ) 
between the S.D.'s of z-scores in the two samples. 

Insert Table 3 about here 

Using the reliability data obtained for the groups, we can estimate 
standard errors of measurement for the individual z-scores and for the 
averaged z-scores. These standard errors of measurement can also be 
translated in terms of SFI units. The relevant results are given in 
Table 3 for the two groups, and for the two groups combined. Some 
explanation is in order. In the case of the z-scores derived for an 
individual's judgment for a particular word, the reliabilities and 
mean square errors are those computed by Ebel's (1951) method. The 
standard errors of measurement in z-score units are the square roots 
of the mean square error values. To translate these into terns of SFI 
units, it 1* assumed that the correlation between z-scores and SFI values 
is unity; since the z-scores for any given individual have an S.D. of 
unity, we multiply the standard errors of measurement by the S.D. of 
the SFI values, which is 13. 896. 
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In the: case of the averaged z_-scores, the reliabilities are those 

obtained by the Ebel method. The standard errors of measurement are 

then obtained by the formula ® ,E, measm t = 0 z'^~ ~ w ^ ere ° z * s S’ 0, 
of the obtained averaged z- scores for the words. Translation of these 
standard errors of measurement into SFI units again assumes perfect 
correlation between the SKE values and the averaged z-scores; however, 
since the S.D.'s of these averaged z-scores are not unity, we multiply 
the standard errors of measurement by the ratio of the respective standard 
deviations, i.e., 13.896/a . 

In all cases the 95$ confidence limits are obtained by multiplying 
the standard errors of measurement by + 1.960 to include 9 5$ °f the 
area in the normal curve. 

These results imply that if we regard an individual's judgment of 
the frequency of a particular word as an independently valid measure, 
and translate it into SFI units by the formula: SFI = 56.3 + 13- 9z, 
the resulting value will be within + 9*95 or + 13.85 of the true value 
95$> of the time, depending on whether he is a lexicographer or a 
nonlexicographer, respectively. Roughly speaking, this means that an 
individual can usually judge word-frequency within one order of magnitude. 

Similarly, if we regard the avera ge z-score value given to a word 
by a group such as our samples as an independently valid measure of 
frequency, and translate it into SFI units by the formula SFI 56.3 13 ‘9 

the resulting value will be within + 2.72 or 1 4.35 of the "true” value 
95 <s£ of the time, for groups of lexicographers and nonlexicographers, 
respectively. This degree of accuracy is attained when data are averaged 
over about l4 raters, but even higher reliability coefficients and 
tighter confidence limits are obtained by using data averaged over all 
28 raters, as shown in Table 2. 
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It is of interest, of course, to consider to what degree the 
individual and averaged estimates (in terms of both log values and 
z-score values) are actually correlated with SFI values. First consider 
the correlations for £— scores derived for particular individuals. (la 
this case the correlations with SFI are the same for log values and for 
z-scores since the latter are linear transforms of the former.) Lexicog- 
raphers' subjective estimates tended to be more highly correlated with 
objective frequency ratings than those of the other sample. For 
them, the correlations ranged from .781 "to .972 and had a median of 
.921, whereas for the other adults, the correlations ranged from .657 
to .918 and had a median of . 827 ; these medians are significantly 
different by the Mann-Whitney test ( z ~ 3-25, £<.0l). 

The two procedures of averaging individual ratings gave only 
slightly different results. When individual log values were averaged, 
the correlations with SFI values were .97^- an ^ -929 i n the two samples, 
respectively; these are significantly different by Fisher's test 
(t = 2.76, df = 114, £ < .01). The correlations for averaged z-scores 
were .970 and .923, respectively, again significantly different 
( t = 2.53, 2 < .02). For data averaged over the combined samples, the 
correlations; were .966 for log values and .959 f° r averaged z-scores. 

All these correlations, of course, are highly significantly different 
from zero, being based on n = 60. The two sets of scores had an 
intercorrelation of .957 both for average log scores and for average 
z-scores. This result will be remarked on in the discussion. 

A feature of the subjective magnitude estimation procedure is that 
it enables determination of a "psychophysical" relation between objective 
experience and subjective judgment. If we assume Stevens' power law 
to apply in the case of word frequency judgments, the relationship should 
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be of form 

j-al®, 

or, taking logarithms, 

log j. = log a ~r m log f, 

where j. — judged frequency and f = objective frequency, and where ^ and 
f both have arbitrary origins and units. There is interest in the 
magnitude of the exponent m, since it expresses how rapidly subjective 
frequency increases as a function of* experienced fre^^ancy. The 
value of the constant a is, however, dependent solely on the origins 
of the scales in which the measurements are expressed. The present 
data yield some evidence on the values of these constants. 

First, consider the case where the judgment is assumed to be 
perfectly correlated with the experienced frequency. The standard 
deviation of the SFI values of the stimuli used in this experiment was 
13.896, but since the SFI values represent logarithmic values multiplied 
by a factor of ten, we may take the standard deviation of the log 
frequencies as being equal to 1.390* The standard deviations of the 
log values assigned by each individual over the words were obtained. 

The ratios of these standard deviations to the S.D. of* the stimulus 
logs may then be regarded as representing values of m. For Sample 1 
(lexicographers), these ratios ranged from .410 to 2.04l, with a median 
of .917; when the ratio is based on the S.D. of* the averaged log values, 
it becomes .854-. For Sample 2, the individual ratios ranged frcm 
.536 to 1.4-8 with a median at .654-; on the basis of the averaged log 
values the ratio is .685. Thus, on the assumption of* a perfect 
correlation between judgment and experienced frequency our samples 
tend slightly to underestimate ratios of frequencies, since the ideal 
logarithmic ratio would be 1.00. Nevertheless, there are wide 
individual differences, attributable partly to differences in people's 
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perceptions of ratios of word frequencies and partly, perhaps, to 
differences in their habits of handling numbers. 

When the regressions of* log values on SF1 values (divided by 10) 
are considered, the ratios are obtained directly from the slope 
coefficients. Regression equations were obtained for each individual 
and for averaged data. For Sample 1, the ratios ranged from *358 to 
1.946 with a median at .868; on the basis of averaged data, the ratio 
was .832. For Sample 2, the ratios ranged from .379 to 1.227 with 
a median of .507; for averaged data, the coefficient was .581. 

There is no particular interest in the values of (log a.) since 

these values merely indicate the log values that would presumably 

(by extrapolation) have been assigned to a word having a value of 

SFI = 0. It happens that SFI is scaled so that it takes the value 

of 0 when the word probability is 10 , or 1 in 10 billion. Thus, 

these values of (log a) are a function of the general sizes of numbers 

employed by the Ss . However, it is of some interest to note that the 

values of averaged scores corresponding to SFI ~ 0 were -3*677 

Sample 1 and -3.270 for Sample 2, based on the regression equations of 

averaged z-scores on SFI values. These numbers, of course, are well 

outside the range of averaged z-values actually obtained, a result that 

is reasonable in view of the fact that there were no stimulus words 

-10 

as rare as one with a probability of 10 

Requiring each subject to consider the words in a different random 
order made it possible to investigate whether the relative frequency 
of the first word considered made for differences in the accuracy or 
other aspects of the subjective judgments . It was thought that the 
objective relative frequency of that word might influence the number 
that the subject would assign to the word, and in that way also 
influence the characteristics of the scale used for the subsequent 
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judgments, and indirectly, the accuracy of those judgments. Over the 
combined samples (N = 28), there was a correlation of .458 ( 2 < .05) 
between the SFI of the first stimulus word and the log value of the 
number assigned to it; the corresponding correlation with the z-score 
derived from that log value was .922 ( 2 < -001). This means that Ss 
tend to choose a "first number" or anchor value in accordance with 
its perceived relative frequency. (This is not always the case; one 
S assigned "1" to the first word, which was other, with SFI - 72.2, and 
still managed to render highly accurate judgments hy using mainly 

fractional numbers for the remaining words.) The log of the "first 

number” chosen hy the Ss was significantly correlated (r = .655, £ < * 01 ) 
with the mean log response to all words, a result that suggests that 
this number tends to establish the scale on which the remaining words 
are judged. Nevertheless, the log magnitude of that number was not 
significantly correlated (r = .122, £> .05, with no evidence of 
significant curvilinearity) with an S’s accuracy in estimating objective 
frequencies, when accuracy is indexed by the correlation between the 
log of his subjective judgment and SFI. Nor was the average log value 
of an S’s responses over all stimuli significantly related to his 
accuracy (r = - . 120 ). The standard deviation of the log values of his 
responses, however, was significantly correlated with his accuracy 

(r = .433, P < -05). 

We may conclude from these data that although the objective 
relative frequency of the first word considered has some influence on 
the scale used for the subsequent words, it does not influence the 
accuracy of the judgments. It is immaterial, in the SME task, whether 
the first word judged is a high, medium, or low frequency word. However, 
there is a tendency for Ss who perceive frequency ratios as relatively 
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large to render more accurate estimates than Ss who perceive frequency 
ratios as relatively small. ; 



Discussion 

The data give evidence that the subjective estimates are consistently 
deviant from the objective frequency values. The correlation between 
the two sets of averaged z-scores was .957; the two sets correlated 
with SP 'I values to the extent of .970 and -929, respectively. If we 
partial out the SFI variable, the inter-set correlation stands at 
.657 (df = 57, 2 ^ .001). This result suggests that the subjective 
estimates of frequency are not measuring exactly the same thing as 
the objective data. There arises the question of which of -these measures 
is more valid with respect to "true" word probability. It can be 
argued that the subjective estimates are more valid, on the grounds 
that objective frequency counts such as the Thomdike-Lorge count are 
subject to biases of various kinds in sampling, in establishing units, 
etc. , and that human observers are better able to discount such biases. 
Furthermore, the samples used in objective frequency counts, seldom 
more than a few million tokens, are small in comparison to the number 
of tokens experienced by the human observer over his lifetime. Objective 
frequency counts are particularly subject to bias in the case of wonis 
of low probability. If the true probability of a word is less than l/N, 
where N is the number of tokens in a sample, and if that word appears 
at all in the sample, by chance, its observed probability is at least 
l/N, a value that is biased upward. For example, a word with a true 
probability of 1 in a million that happens to occur once in a sample of 
1000 words will have an observed probability of 1 in a thousand. In 
the present data, it can be observed by plotting z-scores against SFI 
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values that particularly in the case of low frequency words, the 
z-scores are relatively lower than the SFI values in relation to the 
line of equivalence (the line of equal standard deviation scores for 
the two variables). Thus, the argument that subjective estimates 
are more valid than objective data tends to be stronger in the case 
of low frequency words. 

Ultimately, the question of the validity of subjective estimates 
can probably be settled only by comparing them with data obtained 
from much larger and more refined frequency counts than are currently 
available. The high degree of precision that can be attained by 
averaging SME data over relatively small numbers of raters recommends 
this technique as likely to be useful in assessing the frequency 
attribute of stimuli in verbal learning experiments. By their very 
nature, subjective estimates reflect perceived frequency and hence have 
more imm ediate psychological relevance than word count data. 




14 



► 

\ 



-13- 

references 

Carroll, J. B. An alternative to Juilland's usage coefficient for 
lexical frequencies, and a proposal for a Standard Frequency 
Index ( SFl ) . Computer Studies in the Humanities and Verbal 
Behavior , in press. 

Ebel, R. L. Estimation of the reliability of ratings. Psychometrika, 
1951, 16 , 407-424. 

Kucera, H., & Francis, W. N. Computational analy sis of present-day 

American English . Providence, R.I.: Brown University Press, 1967* 

Morris, W. (Ed.) The American Heritage dictionary of the English 

language. New York and Boston: American Heritage Pub. Co. and 

Houghton Mifflin, 19^9- 

Shapiro, B. J. The subjective scaling of relative word frequency. 
(Doctoral dissertation, Harvard University) Cambridge, Mass.: 
University Microfilms, 1967* No. 67-12,549* 

Shapiro, B. J. The subjective estimation of relative word frequency. 

Journal of Verbal Learning and Verbal Behavior , 19 & 9 , 8, 248-251. 
Stevens, S. S. The direct estimation of sensory magnitudes— loudness . 

American Journal of Psychology ! 195^> 1-25* 

Stevens, S. S. Problems and methods of psychophysics. Psychologica l 

Bulletin . 1958? 177-198* 

Thorndike, E. L., & Lorge, I. The teach er's word book of 3°t 0QQ word s.. 
New York: Teachers College Press, 1944. 




15 



FOOTNOTE 



"^This work was supported by the National Institute oi Child 
Health and Human Development, under Research Grant 1 P 01 HDO1762. I 
thank Mr. Barry Richman, of the American Heritage Publishing Co., Inc., 
for securing the cooperation of both groups of subjects and for arranging 
for the collection of the data. I also thank Miss Barbara Witten for 
her assistance in the computations and data analysis. 



Table 1 



Word 

the 

of 

and 

that 

as 

you 

when 

other 

after 

each 

few 

again 

night 

next 

early 

half 

result 

music 

final 

list 

couple 

price 

actual 

base 

spread 

address 

scale 

suit 

humor 

swift 

victim 

anchor 

convert 

charter 

stride 

switch 

volcano 

heritage 

superb 

skirmish 

cloister 

dissent 

thud 

straggle 

vicar 

shank 

ignite 

cryptic 

veterinary 

modulate 

drivel 

abduct 

torpor 

dill 

ocular 

pachyderm 

spicule 

grout 

echidna 



Mean Log Values, Mean z-Scores , and Standard Deviations of z-Scores, for 
Sample 1, Sample 2, and Combined Samples, for Subjective Magnitude 
Estimates of 60 Words, with SFI Values of the Words 



Sample 1 ( N = 


15) 


Sample 2 ( N = 


13) 


Combined Samples (N = 28) 


Mean Mean 


S.D . 


Mean Mean 


S.D. 


Mean Mean S.D. 



SFI 


Log 


z 


z 


Log 


z 


z 


Log 


z 


z 


88. U 


1+.95 


2.01 


.^75 


1+.57 


1.1*6 


.1*1*6 


4.77 


1.75 


.537 


85.6 


U.59 


1.68 


.318 


1* . 30 


1.23 


.31*5 


1+.1+6 


1.1*7 


.1*00 


8U.5 


1+.6 9 


1.79 


.327 


1+.1+8 


1.1*0 


.300 


4 . 59 


1.6l 


.370 


80.2 


1+.3U 


1.1+7 


. 3'+l 


1* .20 


1.15 


.2U6 


1* .28 


1.32 


.31*0 


78.5 


U.18 


1.39 


.279 


l+.lU 


1.09 


.1*00 


4.l6 


1.25 


.372 


77.2 


l*.l6 


1.37 


.328 


1*. 07 


1.09 


.1*00 


1* . 12 


1.24 


.389 


75.1 


1+.31 


1.1*1* 


.301+ 


1+ .20 


1.06 


.509 


1+.26 


1.26 


.454 


73.6 


3.95 


1.16 


.31+0 


1+.07 


•97 


.557 


1+.01 


1.07 


.1*61* 


72.2 


3.73 


1.02 


.1+00 


3.90 


.82 


.1*02 


3.81 


.93 


.1*13 


70.2 


3.68 


.98 


.288 


3.96 


.89 


.289 


3.81 


.91* 


.292 


69. U 


3.70 


.98 


.261+ 


3.98 


•92 


.377 


3.83 


.95 


.323 


67-7 


3.23 


.69 


.330 


3.5 1 * 


.63 


.1*30 


3.37 


.66 


.381 


67.6 


3.57 


.92 


.251* 


3.82 


.80 


.206 


3.69 


.86 


.21+0 


66.1 


3.07 


.52 


.252 


3.61 


.57 


.381 


3.32 


.51* 


.319 


65.9 


3.35 


•7^ 


.216 


3.91 


.83 


.278 


3.61 


.78 


.251 


65.6 


3.09 


.56 


.31*1 


3.35 


.1*1* 


.372 


3.21 


.50 


.361 


61*. 3 


3.26 


.71 


.338 


3.1*5 


.1*7 


.21*7 


3.35 


.60 


.322 


63.8 


2.97 


.1+0 


.327 


3.1*1 


.1*5 


.21*1 


3.17 


.1*2 


.291 


63.3 


2.82 


.32 


.362 


3.1*3 


.37 


.523 


3.10 


.31* 


.1*1*5 


61.9 


2.82 


.32 


.1+10 


3.35 


.38 


.21*8 


3.07 


.35 


.31*6 


61.2 


2.65 


.21 


.295 


3.63 


.53 


.1*1*1* 


3.10 


.36 


.1+01+ 


60.8 


2.82 


.1*2 


.331 


3.55 


.51 


.309 


3.16 


.1*6 


.321* 


60.3 


2.77 


.31 


.331 


3.65 


.59 


.31*1 


3.18 


.1*1* 


.361* 


60.0 


2.59 


.20 


.1*18 


3.35 


.^5 


A73 


2.9I+ 


•32 


.1*62 


59.5 


2.66 


.20 


.227 


3.34 


.36 


.627 


2.98 


.27 


.465 


59.1 


2.61 


.19 


• 33^ 


3-24 


.22 


.1*22 


2.90 


.20 


.378 


58.8 


2.63 


.21 


.275 


3.37 


.31 


• 333 


2.97 


.26 


.307 


57.7 


2.38 


.03 


.251 


2.99 


.11 


.303 


2.66 


.07 


.279 


56.7 


2.69 


.23 


.21*9 


3.35 


.1*0 


.321 


3.00 


.31 


.297 


56.6 


2.5 1 * 


.11 


.21*2 


3.18 


.15 


.396 


2.8U 


.13 


.323 


56.3 


2.25 


- .09 


.299 


2.97 


.11+ 


.1*50 


2.58 


.02 


.394 


55.6 


2.26 


- .15 


.31*8 


3.09 


.22 


.550 


2.65 


.02 


.1*89 


5U.1 


1.92 


- .33 


.215 


2.67 


- .15 


.675 


2.27 


- .25 


.1*91* 


5I+.0 


2.13 


- .22 


.21+1 


3.02 


.00 


.1*58 


2.5I+ 


- .12 


.375 


53.0 


1.86 


- .1*0 


.177 


2.61* 


- .22 


.538 


2.22 


- .32 


.399 


52.6 


1.97 


- .31 


.226 


2.39 


- .1*1* 


.1*93 


2.16 


- .37 


.380 


51.8 


2. U9 


.06 


.201* 


2.98 


.07 


.336 


2.72 


.06 


.273 


51.5 


1.81 


- .1*1* 


.281 


2.37 


- .56 


.1*59 


2.07 


- .50 


.379 


1+9.0 


1.80 


- .1*6 


.286 


2.1*8 


- .38 


.352 


2.12 


- .1*2 


.321 


1+9-0 


2.2l* 


- .13 


.357 


2.88 


.00 


.1*20 


2.51+ 


- .07 


.393 


1+8.5 


1.62 


- .61 


.259 


2.5I+ 


- .36 


.505 


2.05 


- .1*9 


.1*12 


U7. 8 


1.39 


- .78 


.265 


2.10 


- .80 


.600 


1.72 


- .79 


.1*53 


1+6.0 


2.03 


- .29 


.389 


3.08 


.10 


.1*57 


2.52 


- .11 


.1*65 


1+6.0 


1.62 


- .51* 


.316 


2.1*6 


- .1+1+ 


.1*19 


2.01 


- .1*9 


.371 


1+1*. 8 


1.62 


- .53 


.390 


1.99 


- .82 


.906 


1.79 


- .66 


.695 


1+1*. 8 


.96 


-1.13 


.351 


1.99 


- .81* 


.1*1*5 


1.1*1* 


-1.00 


.1*23 


1*3.0 


1.21 


- .86 


.31*0 


2.02 


- .72 


• 521 


1.59 


- .80 


.1*39 


1*3. 0 


1.69 


- .51* 


.226 


2.69 


- .20 


.358 


2.15 


- .38 


.31*0 


1+0.0 


1.1*1 


- .79 


.205 


2.23 


- .71 


.1*1*0 


1.79 


- .75 


.338 


1+0.0 


1.1*0 


- .72 


.381* 


2.18 


- .66 


.1*71 


1.76 


- .79 


.1*28 


39.8 


1.32 


- .85 


.358 


2.18 


- .69 


.721* 


1.72 


- .78 


.561* 


39.8 


1.32 


- .82 


.1*23 


1.05 


-1.81 


.51*0 


1.19 


-1.28 


.689 


38.9 


1.51 


- .69 


.261 


2.38 


- .1*5 


.556 


1.91 


- .58 


.1*1*1 


38.9 


1.21 


- .95 


.311 


1.12 


-1.68 


.81*9 


1.17 


-1.29 


.720 


38.2 


1.11 


- .98 


.681 


2.23 


- .60 


.389 


1.63 


- .80 


.596 


37.9 


.90 


-1.23 


.1*73 


1.81* 


-1.06 


.588 


1.3l* 


-1.15 


.536 


35.9 


.37 


-1.68 


.1*62 


1.1*2 


-1.50 


.968 


.86 


-I.60 


.71*7 


35.9 


-.08 


-2.01 


.513 


.76 


-2.08 


.1*72 


.31 


-2.0U 


.1*96 


35.2 


-.03 


-2.01+ 


.616 


l.ll* 


-1.71 


.802 


.51 


-1.89 


.727 


33.5 


.01 


-2.05 


• 719 


.53 


-2.32 


.573 


.25 


-2.18 


.669 
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Table 2 

Reliabilities, Standard Errors of Measurement, and 
Confidence Limits for Individual and Averaged Estimates 



Individual z-Scores for Words 


Sample 1 

(Lexicographers ) 

(N= 15) 


Sample 2 
(Other adults) 
(N = 13) 


Combined 
Samples 
(N =- 28 ) 


Reliability- 


.869 


.746 


.801 


Mean square error 


.134 


.259 


.203 


S.E. . (z-scores) 

measmt — 


.366 


.509 


.450 


S.E. x (in SFI units) 

measmt 


5.079 


7.068 


6.253 


95 <jo Confidence limits (SFI units) 


±9.955 


± 13.852 


± 12.256 


Averaged z-Scores 








Reliability 


.990 


.974 


.991 


a 

z 


.937 


.875 


.899 


S .E. , 

measmt 


.098 


.140 


.084 


S.E. , (in SFI units) 

measmt ' 


1.390 


2.224 


1.303 


95% Confidence limits (SFI units) 


±2.724 


±4.359 


±2.555 
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Table 3 





Mean Standard Deviations 


of z^-S cores, 


by Ranges of 


SFI 




No. of 


Mean 


Mean a 


of z-s cores 


SFI 


Words 


SFI 


Sample 1 


Sample 2 


80 - 89.9 


4 


84.7 


.365 


.334 


70-79.9 


6 


74.5 


.323 


.426 


60-69.9 


14 


64.1 


.319 


.348 


50-59.9 


14 


55.5 


.255 


.454 


40-49.9 


12 


45.1 


.314 


.491 


30-39.9 


10 


37.4 


.482 


.646 
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